Video-Audio Domain Generalization via Confounder Disentanglement

Existing video-audio understanding models are trained and evaluated in an intra-domain setting, facing performance degeneration real-world applications where multiple domains distribution shifts naturally exist. The key to domain generalization (VADG) lies alleviating spurious correlations over multi-modal features. To achieve this goal, we resort causal theory attribute such correlation confounders affecting both features labels. We propose a DeVADG framework that conducts uni-modal cross-modal deconfounding through back-door adjustment. performs disentanglement obtains fine-grained at class-level domain-level using half-sibling regression unpaired transformation, which essentially identifies domain-variant factors class-shared cause between false promote VADG research, collect VADG-Action dataset for action recognition with 5,000 video clips across four (e.g., cartoon game) ten classes cooking riding). conduct extensive experiments, i.e., multi-source DG, single-source qualitative analysis, validating the rationality of our analysis effectiveness framework.